Real-Time Human Motion Capture with Multiple Depth Cameras

Commonly used human motion capture systems require intrusive attachment of markers that are visually tracked with multiple cameras. In this work we present an efficient and inexpensive solution to markerless motion capture using only a few Kinect sensors. Unlike the previous work on 3d pose estimation using a single depth camera, we relax constraints on the camera location and do not assume a co-operative user. We apply recent image segmentation techniques to depth images and use curriculum learning to train our system on purely synthetic data. Our method accurately localizes body parts without requiring an explicit shape model. The body joint locations are then recovered by combining evidence from multiple views in real-time. We also introduce a dataset of ~6 million synthetic depth frames for pose estimation from multiple cameras and exceed state-of-the-art results on the Berkeley MHAD dataset.Comment: Accepted to computer robot vision 201

Play and Learn: Using Video Games to Train Computer Vision Models

Video games are a compelling source of annotated data as they can readily provide fine-grained groundtruth for diverse tasks. However, it is not clear whether the synthetically generated data has enough resemblance to the real-world images to improve the performance of computer vision models in practice. We present experiments assessing the effectiveness on real-world data of systems trained on synthetic RGB images that are extracted from a video game. We collected over 60000 synthetic samples from a modern video game with similar conditions to the real-world CamVid and Cityscapes datasets. We provide several experiments to demonstrate that the synthetically generated RGB images can be used to improve the performance of deep neural networks on both image segmentation and depth estimation. These results show that a convolutional network trained on synthetic data achieves a similar test error to a network that is trained on real-world data for dense image classification. Furthermore, the synthetically generated RGB images can provide similar or better results compared to the real-world datasets if a simple domain adaptation technique is applied. Our results suggest that collaboration with game developers for an accessible interface to gather data is potentially a fruitful direction for future work in computer vision.Comment: To appear in the British Machine Vision Conference (BMVC), September 2016. -v2: fixed a typo in the reference

A field dislocation mechanics approach to emergent properties in two-phase nickel-based superalloys

The objective of this study is the development of a theoretical framework for treating the flow stress response of two-phase alloys as emergent behaviour arising from fundamental dislocation interactions. To this end a field dislocation mechanics (FDM) formulation has been developed to model heterogeneous slip within a computational domain representative of a two-phase nickel-based superalloy crystal at elevated temperature. A transport equation for the statistically stored dislocation (SSD) field is presented and implemented within a plane strain finite element scheme. Elastic interactions between dislocations and the microstructure are explicitly accounted for in this formulation. The theory has been supplemented with constitutive rules for dislocation glide and climb, as well as local cutting conditions for the γ’ particles by the dislocation field. Numerical simulations show that γ’ precipitates reduced the effective dislocation mobility by both acting as discrete slip barriers and providing a drag effect through line tension. The effect of varying microstructural parameters on the crystal deformation behaviour is investigated for simple shear loading boundary conditions. It is demonstrated that slip band propagation can be simulated by the proposed FDM approach. Emergent behaviour is predicted and includes: domain size yield dependence (Hall-Petch relationship), γ’ volume fraction yield dependence (along with more complex γ’ dispersion-related yield and post-yield flow stress phenomena), and hardening related to dislocation source distribution at the grain boundary. From these simulations, scaling laws are derived. Also, the emergence of internal back stresses associated with non-homogeneous plastic deformation is predicted. Prediction of these back stresses, due to sub-grain stress partitioning across elastic/plastic zones, is an important result which can provide useful information for the calibration of phenomenological macroscale models. Validation for the presented model is provided through comparison to experimental micro-shear tests that can be found in published literature

Exploiting Points and Lines in Regression Forests for RGB-D Camera Relocalization

Camera relocalization plays a vital role in many robotics and computer vision tasks, such as global localization, recovery from tracking failure and loop closure detection. Recent random forests based methods exploit randomly sampled pixel comparison features to predict 3D world locations for 2D image locations to guide the camera pose optimization. However, these image features are only sampled randomly in the images, without considering the spatial structures or geometric information, leading to large errors or failure cases with the existence of poorly textured areas or in motion blur. Line segment features are more robust in these environments. In this work, we propose to jointly exploit points and lines within the framework of uncertainty driven regression forests. The proposed approach is thoroughly evaluated on three publicly available datasets against several strong state-of-the-art baselines in terms of several different error metrics. Experimental results prove the efficacy of our method, showing superior or on-par state-of-the-art performance.Comment: published as a conference paper at 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS